Method and apparatus for preprocessing fingerprint image

ABSTRACT

Provided in a fingerprint image preprocessing method including receiving an input fingerprint image, performing a short-time Fourier transform (STFT) on the input fingerprint image to obtain a transformed fingerprint image, comparing the input fingerprint image and the transformed fingerprint image, and generating a combined image by combining the input fingerprint image and the transformed fingerprint image based on a result of the comparing.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.16/671,629, filed Nov. 1, 2019 in the U.S. Patent and Trademark Office,which claims priority from Korean Patent Application No. 10-2018-0158508filed on Dec. 10, 2018 in the Korean Intellectual Property Office, thedisclosure of which is incorporated herein by reference in its entirety.

BACKGROUND 1. Field

Example embodiments of the present disclosure relate to fingerprintverification. More particularly, example embodiments relate to a methodof preprocessing a fingerprint image in a fingerprint verificationprocess to improve a quality of the fingerprint image.

2. Description of Related Art

Biometric authentication is used to authenticate a user by using theuser's biological features, such as, fingerprints, irises, voice, facialfeatures, blood vessels, and the like. Such biological features used inuser authentication vary from person to person and rarely change duringthe lifetime of a user. Further, the biological features pose a low riskof theft or imitation, providing high-security authentication. Unlikefobs and other external objects, individuals do not need to exert anyefforts to carry around such features at all times and may thus notsuffer any inconvenience in using the biological features. As one way ofthe biometric authentication, fingerprint verification approaches aremost commonly used due to their high level of convenience, security, andeconomic efficiency. One of these approaches may include comparing afingerprint image of a user requesting user authentication to apreviously registered fingerprint image, and determining whether toauthenticate the user based on a result of the comparing.

Such fingerprint verification approach is most generally used forbiometric authentication, and the need for achieving a high level ofperformance in fingerprint verification is thus on a steady rise. Aquality of a fingerprint image may be degraded due to a poor fingerprintcondition, for example, a dry or wet fingerprint, a restriction of afingerprint sensor, and other causes. The degraded quality may be a maincause of degraded performance in fingerprint verification. There is aneed to improve the quality of a fingerprint image to achieve a highlevel of performance in verification.

SUMMARY

According to an aspect of an example embodiment, there is provided afingerprint image preprocessing method including receiving an inputfingerprint image, performing a short-time Fourier transform (STFT) onthe input fingerprint image to obtain a transformed fingerprint image,comparing the input fingerprint image and the transformed fingerprintimage, and generating a combined image by combining the inputfingerprint image and the transformed fingerprint image based on aresult of the comparing.

The generating of the combined image may include separating a phase anda magnitude from each of the input fingerprint image and the transformedfingerprint image, and applying a result of combining the phase and themagnitude of the input fingerprint image and the magnitude of thetransformed fingerprint image to each area of the combined image, orapplying the transformed fingerprint image to each area of the combinedimage.

The comparing may further include comparing a sensitivity of the inputfingerprint image and a sensitivity of the transformed fingerprint imagefor each corresponding area.

The comparing may further include setting a mask in each of the inputfingerprint image and the transformed fingerprint image, and comparingeach of a plurality of pixels in an area in the input fingerprint imagecorresponding to the mask and each of a plurality of pixels in an areain the transformed fingerprint image corresponding to the mask.

The comparing of each of the plurality of pixels may include comparing apixel value of a pixel in the area in the input fingerprint image and apixel value of a pixel in the area in the transformed fingerprint image.

The comparing of each of the plurality of pixels may include in responseto a pixel value of a first pixel in the area in the input fingerprintimage being greater than a pixel value of a second pixel correspondingto a position of the first pixel in the area in the transformedfingerprint image, allocating a first identifier to a correspondingposition of the first pixel in the combined image, and in response tothe pixel value of the first pixel in the area in the input fingerprintimage being less than the pixel value of the second pixel correspondingto the position of the first pixel in the area in the transformedfingerprint image, allocating a second identifier to the correspondingposition of the first pixel in the combined image.

The generating of the combined image may include in response to thesensitivity of the input fingerprint image being greater than thesensitivity of the transformed fingerprint image, combining the inputfingerprint image and the transformed fingerprint image, and applying aresult of the combining to generate the combined image.

The generating of the combined image may include in response to a pixelvalue of a first pixel in the area in the input fingerprint image beinggreater than a pixel value of a second pixel corresponding to a positionof the first pixel in the area in the transformed fingerprint image,applying, to a corresponding position of the first pixel in the combinedimage, a result of combining the input fingerprint image and thetransformed fingerprint image.

The generating of the combined image may include applying, to thecorresponding position of the first pixel in the combined image, aresult of combining the input fingerprint image and the transformedfingerprint image based on the first identifier, and applying, to thecorresponding position of the first pixel in the combined image, thetransformed fingerprint image based on the second identifier.

A non-transitory computer-readable storage medium storing instructionsthat, when executed by a processor, may cause the processor to performthe fingerprint image preprocessing method.

According to an aspect of an example embodiment, there is provided afingerprint image preprocessing apparatus including at least oneprocessor, and a memory configured to store an instruction to operatethe processor, wherein the processor is configured to receive an inputfingerprint image, perform a short-time Fourier transform (STFT) on theinput fingerprint image to obtain a transformed fingerprint image,compare the input fingerprint image and the transformed fingerprintimage, and generate a combined image by combining the input fingerprintimage and the transformed fingerprint image based on a result of thecomparing.

The processor may be further configured to separate a phase and amagnitude from each of the input fingerprint image and the transformedfingerprint image, and apply an image resulting from of combining thephase and the magnitude of the input fingerprint image and the magnitudeof the transformed fingerprint image to each area of the combined image,or apply the transformed fingerprint image to each area of the combinedimage.

The processor may be further configured to set a mask in each of theinput fingerprint image and the transformed fingerprint image, andcompare each of a plurality of pixels in an area in the inputfingerprint image corresponding to the mask and each of a plurality ofpixels in an area in the transformed fingerprint image corresponding tothe mask.

The processor may be further configured to compare a pixel value of apixel in the area in the input fingerprint image and a pixel value of apixel in the area in the transformed fingerprint image.

The processor may be further configured to in response to a pixel valueof a first pixel in an area in the input fingerprint image being greaterthan a pixel value of a second pixel corresponding to a position of thefirst pixel in an area in the transformed fingerprint image, allocate afirst identifier to a corresponding position of the first pixel in thecombined image, and in response to the pixel value of the first pixel inthe area in the input fingerprint image being less than the pixel valueof the second pixel corresponding to the position of the first pixel inthe area in the transformed fingerprint image, allocate a secondidentifier to the corresponding position of the first pixel in thecombined image.

The processor may be further configured to apply, to the correspondingposition of the first pixel in the combined image, a result of combiningthe input fingerprint image and the transformed fingerprint image basedon the first identifier, and apply, to the corresponding position of thefirst pixel in the combined image, the transformed fingerprint imagebased on the second identifier

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects will be more apparent by describingcertain example embodiments with reference to the accompanying drawings,in which:

FIG. 1 is a diagram illustrating an example of how a fingerprint isrecognized by a fingerprint image preprocessing apparatus according toan example embodiment;

FIG. 2 is a diagram illustrating an example of how an input fingerprintimage is transformed into a fingerprint image through a short-timeFourier transform (STFT) according to an example embodiment;

FIG. 3 is a flowchart illustrating an example of a fingerprint imagepreprocessing method according to an example embodiment;

FIG. 4 is a diagram illustrating an example of an overall operation of afingerprint image preprocessing method according to an exampleembodiment;

FIG. 5 is a diagram illustrating an example of how a combined image isgenerated from an input fingerprint image through a fingerprint imagepreprocessing method according to an example embodiment;

FIG. 6 is a diagram illustrating an example of how a combined image isobtained from fingerprint images input from various sensors according toan example embodiment; and

FIG. 7 is a diagram illustrating an example of a fingerprint imagepreprocessing apparatus according to an example embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to example embodiments, examples ofwhich are illustrated in the accompanying drawings, wherein likereference numerals refer to the like elements throughout.

The following structural or functional descriptions are exemplary tomerely describe the example embodiments, and the scope of the exampleembodiments is not limited to the descriptions provided in the presentdisclosure. Various changes and modifications can be made thereto bythose of ordinary skill in the art.

Although terms of “first” or “second” are used to explain variouscomponents, the components are not limited to the terms. These termsshould be used only to distinguish one component from another component.For example, a “first” component may be referred to as a “second”component, or similarly, and the “second” component may be referred toas the “first” component within the scope of the right according to theexample embodiments of the present disclosure.

It will be understood that when a component is referred to as being“connected to” another component, the component can be directlyconnected or coupled to the other component or intervening componentsmay be present.

As used herein, the singular forms are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It shouldbe further understood that the terms “comprises,” “comprising,”“includes,” and/or “including,” when used in this specification, specifythe presence of stated features, integers, steps, operations, elements,components or a combination thereof, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list. Forexample, the expression, “at least one of a, b, and c,” should beunderstood as including only a, only b, only c, both a and b, both a andc, both b and c, or all of a, b, and c.

Unless otherwise defined herein, all terms used herein includingtechnical or scientific terms have the same meanings as those generallyunderstood by one of ordinary skill in the art. Terms defined indictionaries generally used should be construed to have meaningsmatching with contextual meanings in the related art and are not to beconstrued as an ideal or excessively formal meaning unless otherwisedefined herein.

Hereinafter, examples will be described in detail with reference to theaccompanying drawings, and like reference numerals in the drawings referto like elements throughout.

FIG. 1 is a diagram illustrating an example of how a fingerprint isrecognized by a fingerprint image preprocessing apparatus according toan example embodiment. A fingerprint image preprocessing apparatus maybe an apparatus for preprocessing a fingerprint image, and a fingerprintimage preprocessing method may be a method of preprocessing afingerprint image.

Referring to FIG. 1, a fingerprint image preprocessing apparatus 100obtains a transformed fingerprint image 107 by preprocessing an inputfingerprint image 105 of a user 101, and compares the input fingerprintimage 105 and the transformed fingerprint image 107. The fingerprintimage preprocessing apparatus 100 combines the input fingerprint image105 and the transformed fingerprint image 107 by selecting an image witha higher sensitivity between the input fingerprint image 105 and thetransformed fingerprint image 107 for each corresponding area, andgenerates a combined image 109. The combined image 109 is then used toextract feature points to be used for fingerprint verification.

The fingerprint verification may be a verification method used todetermine whether the user 101 attempting at such verification is avalid user or not, and verify a valid user in applications, such as, forexample, user log-in, payment services, financial services, and accesscontrol. Referring to FIG. 1, a fingerprint verification apparatus thatperforms such a verification method includes the fingerprint imagepreprocessing apparatus 100, and is included in, or represented by, acomputing apparatus. The computing apparatus may include various typesof products such as, for example, a smart phone, a wearable device, apersonal computer (PC), a tablet PC or simply tablet, a desktop, alaptop, a notebook, a personal digital assistant (PDA), a set-top box, ahome appliance, a biometrics-based door lock, a security device, a smartvehicle, and others capable of wireless communication or networkcommunication consistent with that disclosed herein.

A fingerprint includes ridges and valleys. A ridge indicates a convexportion of protruding pores, and a valley indicates a relatively concaveportion between ridges. A point at which a ridge ends is referred to asan ending point, and a point at which a ridge is bifurcated is referredto as a bifurcation point. A ridge includes a beginning and an end ofthe ridge, a bifurcation, a core, and a delta. The core indicates an endportion of a rotation of a ridge, and the delta indicates a point atwhich flows of ridges converge in three directions. These are alsofinely classified into a dot, a right ending, an island or isolatedportion, a bridge, a short ridge, and the like, which are collectivelyreferred to as minutiae. Each of the elements included in a fingerprintdescribed above is also individually referred to as a feature.

As illustrated, the computing apparatus determines whether the user 101attempting to have access to the computing apparatus is a valid user ornot by analyzing a fingerprint pattern of the input fingerprint image105 sensed through a fingerprint sensor 103. For example, when the user101 inputs a fingerprint of the user 101 to unlock the computingapparatus, the computing apparatus may compare the input fingerprintimage 105 obtained from the input fingerprint through the fingerprintsensor 103 to at least one registered fingerprint images stored in adatabase (DB), and determine whether to unlock the computing apparatusbased on a result of the comparing. The DB may include thereinregistered fingerprint images of one or more fingers.

For example, a valid user may register fingerprint information of thevalid user in advance in the computing apparatus, and the computingapparatus may store the registered fingerprint information in the DB ora cloud storage. The registered fingerprint information may be stored ina form of registered fingerprint image. In a fingerprint registrationprocess, a user may register various fingerprint images.

Before extracting a feature point from an input fingerprint image, thecomputing apparatus may preprocess the input fingerprint image to moreeffectively extract the feature point. The computing apparatus mayextract the feature point from a fingerprint image transformed throughthe preprocessing. The computing apparatus may then compare the featurepoint of the input fingerprint image obtained through a fingerprintsensor and each of feature points of previously registered fingerprintimages.

However, the fingerprint image transformed through the preprocessing maynot include a feature point that is sufficient to identify afingerprint. When fingerprint verification is performed by comparing thetransformed fingerprint image with such an insufficient feature pointand a registered fingerprint image one to one, an undesirable resultsuch as false acceptance may occur. For example, when the fingerprintimage transformed through the preprocessing does not sufficientlyreflect the feature point included in the input fingerprint image, asimilarity between the input fingerprint image and a registeredfingerprint image may be set to be high due to a simple fingerprintpattern and such a result of false acceptance may be highly likely tooccur.

The false acceptance in fingerprint verification may adversely affectaccuracy in the fingerprint verification or a recognition rate of thefingerprint verification, and it is thus desirable to prevent such anoccurrence of the false acceptance. Thus, example embodiments describedherein provide a preprocessing method used to more effectively extract afeature point from a fingerprint image while preventing the falseacceptance.

According to an example embodiment, the fingerprint image preprocessingapparatus 100 uses both the input fingerprint image 105 and thetransformed fingerprint image 107, and thus applies a feature of such anoriginal image and a feature emphasized by the preprocessing to thecombined image 109. Thus, the fingerprint image preprocessing apparatus100 may obtain an image including a feature point more sufficient forfingerprint verification, and the fingerprint verification may be moreaccurately performed.

The fingerprint image preprocessing apparatus 100 preprocesses the inputfingerprint image 105 using a short-time Fourier transform (STFT) alongwith frequency-domain magnitude and phase features. The fingerprintimage preprocessing apparatus 100 may thus obtain a more effectivepreprocessed image by combining a feature of a fingerprint imagetransformed through the STFT and the frequency-domain magnitude andphase features. The fingerprint image preprocessing apparatus 100 maycombine a fingerprint image in which a main pattern is emphasized by theSTFT and a fingerprint image in which an auxiliary pattern is emphasizedby a frequency-domain magnitude and phase analysis. To the combinedimage 109, both the main pattern and the auxiliary pattern may beapplied. The main pattern may indicate an overall feature or shape of afingerprint formed by ridges and valleys, and the auxiliary pattern mayindicate all other patterns of the fingerprint excluding the mainpattern. The auxiliary pattern may include, for example, an edge and aline of a ridge. The auxiliary pattern may also be referred to as asubpattem or a small pattern.

In a fingerprint image transformed through the STFT, a ridge and avalley of a fingerprint may be clearer, and other features may beblurred or omitted. That is, an auxiliary pattern of a fingerprint maybe blurred or omitted. The fingerprint image preprocessing apparatus 100may thus obtain a main pattern indicating ridges and valleys from thetransformed fingerprint image 107 obtained through the STFT, and obtainan auxiliary pattern of the ridges from the input fingerprint image 105.To this end, the fingerprint image preprocessing apparatus 100 mayseparate a frequency magnitude component and a frequency phase componentfrom each of the input fingerprint image 105 and the transformedfingerprint image 107 obtained through the STFT.

The fingerprint image preprocessing apparatus 100 may separate amagnitude and a phase of each of the input fingerprint image 105 and thetransformed fingerprint image 107 using a fingerprint transformationsuch as, for example, Fourier transform. A two-dimensional (2D) Fouriertransform and frequency-domain magnitude and phase features are asfollows. A Fourier transform refers to a method of analyzing an image interms of frequency, and indicates which frequency components the imageconsists of. Equation 1 represents how a 2D image f(x, y) is transformedinto a frequency domain through the Fourier transform.

F(u, v)=∫_(−∞) ^(∞)∫_(−∞) ^(∞) f(x, y)e ^(−f2π(ux+vy)) dx dy   [Equation1]

In Equation 1, F(u, v) is represented by a complex number as in Equation2, and a magnitude and a phase in the frequency domain are representedby Equation 3.

F(u, v)=F _(R)(u, v)+jF _(I)(u, v)   [Equation 2]

magnitude|F(u, v)|phase arctan(F _(I)(u, v)/F _(R)(u, v))   [Equation 3]

In an image transformed through the Fourier transform, a phase may shiftbasic waveforms and may have a same value at edges and lines accordingto a principle of phase congruency. Herein, a same phase value at theedges or lines may indicate that various frequency waveforms may overlapat a corresponding position. A phase in the frequency domain may beclosely related to an edge or line. A magnitude in the frequency domainmay indicate a signal strength at a corresponding frequency. A frequencymagnitude in an image may affect an overall feature of the imageexcluding an edge or line. Thus, when changing magnitude components inthe frequency domain, such an overall feature such as a texture of theimage may also change.

The fingerprint image preprocessing apparatus 100 may generate thecombined image 190 by combining the separated phase and magnitude. Thefingerprint image preprocessing apparatus 100 may obtain a relativelyclear texture from the transformed fingerprint image 107, and obtain anauxiliary pattern such as an edge and line from the input fingerprintimage 105. The fingerprint image preprocessing apparatus 100 may use thetransformed fingerprint image 107 as a basic image. However, when thereis an insufficient feature of the transformed fingerprint image 107, thefingerprint image preprocessing apparatus 100 may use the inputfingerprint image 105 to generate the combined image 109.

When using the transformed fingerprint image 107 obtained through theSTFT, the auxiliary pattern may be blurred or omitted although the mainpattern may be reflected in the transformed fingerprint image 107. Thus,the fingerprint image preprocessing apparatus 100 may use the inputfingerprint image 105 for an area in which the auxiliary pattern isblurred or omitted by comparing the transformed fingerprint image 107and the input fingerprint image 105. For example, the fingerprint imagepreprocessing apparatus 100 may determine whether to use the transformedfingerprint image 107 or the input fingerprint image 105 based onEquation 4.

A_(mag)

B_(mag)

M_(mag)

m _(i,j)=1(b _(i,j) −a _(i,j)<0)

m _(i,j)=0(b _(i,j) −a _(i,j)>0)   [Equation 4]

Referring to Equation 4, the fingerprint image preprocessing apparatus100 sets a mask M_(mag) of a matrix, m*n. Herein, the mask may be usedto specify an area in an image. In Equation 4, A_(mag) denotes amagnitude component of the input fingerprint image 105 in the matrixm*n, and B_(mag) denotes a magnitude component of the transformedfingerprint image 107 in the matrix m*n. The fingerprint imagepreprocessing apparatus 100 obtains each pixel value a_(i,j) in the maskby applying the mask to the input fingerprint image 105, and obtainseach pixel value b_(i,j) in the mask by applying the mask to thetransformed fingerprint image 107.

The fingerprint image preprocessing apparatus 100 then compares thevalue a_(i,j) and the value b_(i,j). In response to the value a, beinggreater than the value b_(i,j), the fingerprint image preprocessingapparatus 100 determines that the transformed fingerprint image 107 doesnot include the auxiliary pattern included in the input fingerprintimage 105. The fingerprint image preprocessing apparatus 100 allocates 1to a mask value m_(i,j) at a position (i, j) corresponding to the valuea_(i,j) and the value b_(i,j). The value 1 to be allocated may be usedfor identification, and thus other different values may also beallocated.

Conversely, in response to the value a_(i,j) being less than the valueb_(i,j), the fingerprint image preprocessing apparatus 100 determinesthat the transformed fingerprint image 107 includes the auxiliarypattern included in the input fingerprint image 105. The fingerprintimage preprocessing apparatus 100 allocates 0 to the mask value m_(i,j)at the position (i, j) corresponding to the value a_(i,j) and the valueb_(i,j). The value 0 to be allocated may be used for identification, andthus other different values may also be allocated.

When the mask value m_(i,j) corresponding to a pixel at the position (i,j) is 0, the fingerprint image preprocessing apparatus 100 may select avalue of the transformed fingerprint image 107. When the mask valuem_(i,j) corresponding to the pixel at the position (i, j) is 1, thefingerprint image preprocessing apparatus 100 may combine a component ofthe transformed fingerprint image 105 and a component of the inputfingerprint image 105 and apply a result of the combining to generatethe combined image 109. For example, the fingerprint image preprocessingapparatus 100 may combine the components of the transformed fingerprintimage 107 and the input fingerprint image 105 using Equation 5.

F(u, v)=B′ _(mag)(u, v)*exp(−j*A _(phase)(u, v))   [Equation 5]

In Equation 5, A_(phase) denotes a phase component of the inputfingerprint image 105 which is an original image, and B′_(mag) denotes amagnitude component obtained by combining the magnitude component of theinput fingerprint image 105 and the magnitude component of thetransformed fingerprint image 107.

The fingerprint image preprocessing apparatus 100 obtains a final imagethrough an inverse Fourier transform (IFT). For example, the fingerprintimage preprocessing apparatus 100 may obtain the final image usingEquation 6.

f(x, y)=∫_(−∞) ^(∞)∫_(−∞) ^(∞) F(u, v)e ^(f2π(ux+vy)) du dy   [Equation6]

The final image may have the auxiliary pattern while maintaining a clearfeature obtained through the STFT, and may thus be more naturalfingerprint image than the transformed fingerprint image 107 obtainedthrough the STFT.

FIG. 2 is a diagram illustrating an example of how an input fingerprintimage is transformed into a fingerprint image through a STFT accordingto an example embodiment.

There are various methods to improve an image quality of a fingerprintimage. Among these, a generally used method is estimating a frequencyand an orientation of a local ridge of a fingerprint from a spatialdomain or a frequency domain, and applying a contextual filter based onthe estimated frequency and orientation. For example, the method mayinclude estimating a frequency and an orientation of a local fingerprintridge in a frequency domain through an STFT and applying a contextualfilter based on the estimated frequency and orientation to improve animage quality of a fingerprint image.

Referring to FIG. 2, an input fingerprint image 201 is an original imageformed by a sensor. The input fingerprint image 201 includes a mainpattern formed by ridges and valleys of a fingerprint and an auxiliarypattern including edges and lines of the ridges. In the inputfingerprint image 201, an area indicated by a broken line includes adisconnected portion. A ridge includes a component other than asinusoidal wave, and thus an area 205 of the input fingerprint image 201may have a complex feature.

In a transformed fingerprint image 203 obtained through an STFT, themain pattern formed by the ridges and the valleys is emphasized, whereasthe auxiliary pattern is blurred or omitted. In the transformedfingerprint image 203, an area indicated by a broken line includes adisconnected portion. The STFT may be performed to estimate a frequencyand an orientation under an assumption that a ridge corresponds to asingle specific sinusoidal wave, and thus the main pattern of the ridgesmay be emphasized as illustrated in an area 207 of the transformedfingerprint image 203 and the auxiliary pattern in the area 205 of theinput fingerprint image 201 may be blurred or omitted in the area 207 ofthe transformed fingerprint image 203.

FIG. 3 is a flowchart illustrating an example of a fingerprint imagepreprocessing method according to an example embodiment.

Referring to FIG. 3, in operation 301, a fingerprint image preprocessingapparatus receives an input fingerprint image. The fingerprint imagepreprocessing apparatus may receive the input fingerprint imagecorresponding to an original image from a sensor. The input fingerprintimage may include both a feature of a main pattern and a feature of anauxiliary pattern.

In operation 303, the fingerprint image preprocessing apparatus obtainsa transformed fingerprint image by performing an STFT on the inputfingerprint image. The transformed fingerprint image may include a mainpattern that is clearer than that in the input fingerprint image. In thetransformed fingerprint image, the auxiliary pattern included in theinput fingerprint image may be blurred or omitted.

In operation 305, the fingerprint image preprocessing apparatus comparesthe input fingerprint image and the transformed fingerprint image. Thefingerprint image preprocessing apparatus may compare a sensitivity ofthe input fingerprint image and a sensitivity of the transformedfingerprint image in each corresponding area. Herein, the sensitivitymay indicate how well features included in the original image arereflected. For example, when an image includes an auxiliary pattern, theimage may be determined to have a higher sensitivity compared to animage from which such an auxiliary pattern is omitted.

The fingerprint image preprocessing apparatus may set a mask in each ofthe input fingerprint image and the transformed fingerprint image. Thefingerprint image preprocessing apparatus may compare each of pixels inan area in the input fingerprint image corresponding to the mask andeach of pixels in an area in the transformed fingerprint imagecorresponding to the mask. The fingerprint image preprocessing apparatusmay compare a pixel value of a pixel included in the area in the inputfingerprint image and a pixel value of a pixel included in thecorresponding area in the transformed fingerprint image.

For example, when a pixel value of a first pixel included in the area inthe input fingerprint image is greater than a pixel value of a secondpixel corresponding to a position of the first pixel in the area in thetransformed fingerprint image, the fingerprint image preprocessingapparatus may allocate a first identifier to a corresponding position ofthe first pixel in the combined image. In this example, the firstidentifier may be 1.

Conversely, when the pixel value of the first pixel included in the areain the input fingerprint image is less than the pixel value of thesecond pixel included in the area in the transformed fingerprint image,the fingerprint image preprocessing apparatus may allocate a secondidentifier to the corresponding position of the first pixel in thecombined image. In this example, the second identifier may be 0.

In operation 307, the fingerprint image preprocessing apparatusgenerates the combined image by combining the input fingerprint imageand the transformed fingerprint image based on a result of thecomparing. The fingerprint image preprocessing apparatus may separate aphase and a magnitude of the input fingerprint image, and a phase and amagnitude of the transformed fingerprint image. The fingerprint imagepreprocessing apparatus may use a Fourier transform for the separatingof a phase and a magnitude.

The fingerprint image preprocessing apparatus may apply a result ofcombining the phase and the magnitude of the input fingerprint image andthe magnitude of the transformed fingerprint image to each area of thecombined image, or apply the transformed fingerprint image to each areaof the combined image, based on a result of the comparing. Herein, whena sensitivity of the input fingerprint image is greater than asensitivity of the transformed fingerprint image, the fingerprint imagepreprocessing apparatus may combine the input fingerprint image and thetransformed fingerprint image and apply a result of the combining togenerate the combined image.

For example, when the pixel value of the first pixel included in thearea in the input fingerprint image is greater than the pixel value ofthe second pixel corresponding to the position of the first pixel in thearea in the transformed fingerprint image, the fingerprint imagepreprocessing apparatus may apply a result of combining the inputfingerprint image and the transformed fingerprint image to thecorresponding position of the first pixel in the combined image.

The fingerprint image preprocessing apparatus may apply, to thecorresponding position of the first pixel in the combined image, aresult of combining the input fingerprint image and the transformedfingerprint image based on the first identifier. In addition, thefingerprint image preprocessing apparatus may apply, to thecorresponding position of the first pixel in the combined image, thetransformed fingerprint image based on the second identifier.

FIG. 4 is a diagram illustrating an example of an overall operation of afingerprint image preprocessing method according to an exampleembodiment.

Referring to FIG. 4, in operation 401, a fingerprint image preprocessingapparatus obtains a transformed fingerprint image by performing an STFTon an input fingerprint image.

In operation 402, the fingerprint image preprocessing apparatusseparates a frequency phase component and a frequency magnitudecomponent from each of the input fingerprint image and the transformedfingerprint image. For example, the fingerprint image preprocessingapparatus may apply a Fourier transform to separate the frequency phasecomponent and the frequency magnitude component from each of the inputfingerprint image and the transformed fingerprint image.

The fingerprint image preprocessing apparatus may select a component tobe applied to a combined image by comparing the input fingerprint imageand the transformed fingerprint image. The fingerprint imagepreprocessing apparatus may basically select the transformed fingerprintimage for each area of the combined image. However, when a featureincluded in the input fingerprint image is not included in thetransformed fingerprint image, the fingerprint image preprocessingapparatus may apply a component of the input fingerprint image to acorresponding area of the combined image.

When the feature included in the input fingerprint image is not includedin the transformed fingerprint image, the fingerprint imagepreprocessing apparatus performs operations 403 and 404. In operation403, the fingerprint image preprocessing apparatus combines thefrequency magnitude component of the input fingerprint image and thefrequency magnitude component of the transformed fingerprint image. Inoperation 404, the fingerprint image preprocessing apparatus combinesthe combined frequency magnitude component and the frequency phasecomponent of the input fingerprint image.

These operations described above may be performed on each of all areasof the input fingerprint image, and thus the combined image may beobtained. The combined image, which is a final image, may include anauxiliary pattern while maintaining a clearer feature through the STFT,and thus may be a more natural fingerprint image than the transformedfingerprint image itself obtained through the STFT.

FIG. 5 is a diagram illustrating an example of how a combined image isgenerated from an input fingerprint image through a fingerprint imagepreprocessing method according to an example embodiment.

Referring to FIG. 5, an input fingerprint image 511 which is an originalimage obtained from a sensor includes a main pattern formed by ridgesand valleys, and an auxiliary pattern including edges and lines of theridges. For example, the input fingerprint image 511 may include, as theauxiliary pattern, an area in which ridges are disconnected.

A transformed fingerprint image 514 obtained through an STFT includesthe main pattern that is more emphasized compared to the main pattern inthe input fingerprint image 511. The auxiliary pattern may be blurred oromitted in the transformed fingerprint image 514. For example, the areaof the input fingerprint image 511 in which the ridges are disconnectedmay be omitted in the transformed fingerprint image 514. As describedabove, the main pattern may be more emphasized in the transformedfingerprint image 514, whereas the auxiliary pattern included in theinput fingerprint image 511 may be blurred or omitted in the transformedfingerprint image 514.

A fingerprint image preprocessing apparatus may compare a sensitivity ofthe input fingerprint image 511 and a sensitivity of the transformedfingerprint image 514 for each corresponding area. For example, for anarea corresponding to the area in which the ridges are disconnected, thefingerprint image preprocessing apparatus may determine that thetransformed fingerprint image 514 does not include the auxiliary patternof the input fingerprint image 511 in which the ridges are disconnected.The fingerprint image preprocessing apparatus may then apply theauxiliary pattern of the input fingerprint image 511 to a correspondingarea in the transformed fingerprint image 514.

To this end, the fingerprint image preprocessing apparatus may separatea magnitude and a phase of the input fingerprint image 511, and amagnitude and a phase of the transformed fingerprint image 514. Forexample, as illustrated, the fingerprint image preprocessing apparatusseparates a magnitude 512 of the input fingerprint image 511 and a phase513 of the input fingerprint image 511, and a magnitude 515 of thetransformed fingerprint image 514 and a phase 516 of the transformedfingerprint image 514.

In operation 517, the fingerprint image preprocessing apparatus combinesor synthesizes frequency components using the magnitude 512 and thephase 513 of the input fingerprint image 511, and the magnitude 515 ofthe transformed fingerprint image 514. The fingerprint imagepreprocessing apparatus generates a magnitude component by combining themagnitude 512 of the input fingerprint image 511 and the magnitude 515of the transformed fingerprint image 514, and uses the phase 513 of theinput fingerprint image 511 as a phase component. The fingerprint imagepreprocessing apparatus obtains a frequency component of a combinedimage 518 using the generated magnitude component and the phasecomponent. By performing the above operations on all the areas of theinput fingerprint image 511 and the transformed fingerprint image 514,the fingerprint image preprocessing apparatus obtains the combined image518.

FIG. 6 is a diagram illustrating an example of how a combined image isobtained from fingerprint images input from various sensors according toan example embodiment.

A fingerprint image preprocessing apparatus may perform an STFT on inputfingerprint images which are original images received from varioussensors, and obtain respective combined images. In the exampleillustrated in FIG. 6, a solid-line circle in each image is anindication used for the comparison in terms of how well an auxiliarypattern is represented in each image, and a broken-line circle in eachimage is an indication used for the comparison in terms of how naturaleach image is as compared to a corresponding original image.

Referring to FIG. 6, a fingerprint image 613 transformed from an inputfingerprint image 611 obtained from an ultrasonic sensor does notdesirably include therein an auxiliary pattern included in the inputfingerprint image 611. For example, as illustrated, an area in whichridges are disconnected is not included in the transformed fingerprintimage 613. However, the auxiliary pattern included in the inputfingerprint image 611 is included in a combined image 615 obtained bythe fingerprint image preprocessing apparatus.

A fingerprint image 623 transformed from an input fingerprint image 621obtained from the ultrasonic sensor may be less natural because it doesnot include therein an auxiliary pattern included in the inputfingerprint image 621 although it includes clearer ridges. However, theauxiliary pattern included in the input fingerprint image 621 isincluded in a combined image 625 obtained by the fingerprint imagepreprocessing apparatus, and thus the combined image 625 may represent amore natural result.

A fingerprint image 633 transformed from an input fingerprint image 631obtained from a capacitance sensor does not desirably include therein anauxiliary pattern included in the input fingerprint image 631. Forexample, as illustrated, a white auxiliary pattern disconnecting ridgesis not included in the transformed fingerprint image 633. However, theauxiliary pattern included in the input fingerprint image 631 isincluded in a combined image 635 obtained by the fingerprint imagepreprocessing apparatus.

A fingerprint image 643 transformed from an input fingerprint image 641obtained from the capacitance sensor may be less natural because it doesnot include therein an auxiliary pattern included in the inputfingerprint image 641 although it includes clearer ridges and a clearercore. However, the auxiliary pattern included in the input fingerprintimage 641 is included in a combined image 645 obtained by thefingerprint image preprocessing apparatus, and thus the combined image645 may represent a more natural result.

A fingerprint image 653 transformed from an input fingerprint image 651obtained from an optical sensor may not include an auxiliary patternincluded in the input fingerprint image 651. For example, a length of anarea in which ridges are disconnected is represented shorter in thetransformed fingerprint image 653 than the input fingerprint image 651.However, the auxiliary pattern included in the input fingerprint image651 is included at a length corresponding to the input fingerprint image651 in a combined image 655 obtained by the fingerprint imagepreprocessing apparatus.

FIG. 7 is a diagram illustrating an example of a fingerprint imagepreprocessing apparatus according to an example embodiment.

Referring to FIG. 7, a fingerprint image preprocessing apparatus 100includes at least one processor 701 and a memory 703 configured to storetherein an instruction to operate the processor 701. The fingerprintimage preprocessing apparatus 100 may further include an input andoutput (I/O) interface 705.

The processor 701 may receive an input fingerprint image. The processor701 may obtain a transformed fingerprint image by performing an STFT onthe input fingerprint image. The processor 701 may compare the inputfingerprint image and the transformed fingerprint image. The processor701 may generate a combined image by combining the input fingerprintimage and the transformed fingerprint image based on a result of thecomparing.

The processor 701 may separate a magnitude and a phase from each of theinput fingerprint image and the transformed fingerprint image. Theprocessor 701 may apply, to each area of the combined image, a result ofcombining the magnitude and the phase of the input fingerprint image andthe magnitude of the transformed fingerprint image, or apply thetransformed fingerprint image, based on a result of the comparing.

For example, when a pixel value of a first pixel included in an area inthe input fingerprint image is greater than a pixel value of a secondpixel corresponding to a position of the first pixel in an area in thetransformed fingerprint image, the processor 701 may allocate a firstidentifier to a corresponding position of the first pixel in thecombined image. When the pixel value of the first pixel in the area inthe input fingerprint image is less than the pixel value of the secondpixel in the area in the transformed fingerprint image, the processor701 may allocate a second identifier to the corresponding position ofthe first pixel in the combined image.

The processor 701 may apply, to the corresponding position of the firstpixel in the combined image, a result of combining the input fingerprintimage and the transformed fingerprint image based on the firstidentifier. The processor 701 may also apply, to the correspondingposition of the first pixel in the combined image, the transformedfingerprint image based on the second identifier.

The units described herein may be implemented using hardware componentsand software components. For example, the hardware components mayinclude microphones, amplifiers, band-pass filters, audio to digitalconvertors, non-transitory computer memory and processing devices. Aprocessing device may be implemented using one or more general-purposeor special purpose computers, such as, for example, a processor, acontroller and an arithmetic logic unit (ALU), a digital signalprocessor, a microcomputer, a field programmable gate array (FPGA), aprogrammable logic unit (PLU), a microprocessor or any other devicecapable of responding to and executing instructions in a defined manner.The processing device may run an operating system (OS) and one or moresoftware applications that run on the OS. The processing device also mayaccess, store, manipulate, process, and create data in response toexecution of the software. For purpose of simplicity, the description ofa processing device is used as singular; however, one skilled in the artwill appreciated that a processing device may include multipleprocessing elements and multiple types of processing elements. Forexample, a processing device may include multiple processors or aprocessor and a controller. In addition, different processingconfigurations are possible, such a parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, to independently orcollectively instruct or configure the processing device to operate asdesired. Software and data may be embodied permanently or temporarily inany type of machine, component, physical or virtual equipment, computerstorage medium or device, or in a propagated signal wave capable ofproviding instructions or data to or being interpreted by the processingdevice. The software also may be distributed over network coupledcomputer systems so that the software is stored and executed in adistributed fashion. The software and data may be stored by one or morenon-transitory computer readable recording mediums. The non-transitorycomputer readable recording medium may include any data storage devicethat can store data which can be thereafter read by a computer system orprocessing device.

Example embodiments include non-transitory computer-readable mediaincluding program instructions to implement various operations embodiedby a computer. The media may also include, alone or in combination withthe program instructions, data files, data structures, tables, and thelike. The media and program instructions may be those specially designedand constructed for the purposes of example embodiments, or they may beof the kind well known and available to those having skill in thecomputer software arts. Examples of non-transitory computer-readablemedia include magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD ROM disks; magneto-optical mediasuch as floptical disks; and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM) and random-access memory (RAM). Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter. The described hardware devices may beconfigured to act as one or more software modules in order to performthe operations of the above-described example embodiments, or viceversa.

The examples described herein are to be considered in a descriptivesense only, and not for purposes of limitation. Descriptions of featuresor aspects in each example are to be considered as being applicable tosimilar features or aspects in other examples. Suitable results may beachieved if the described techniques are performed in a different order,and/or if components in a described system, architecture, device, orcircuit are combined in a different manner and/or replaced orsupplemented by other components or their equivalents.

While example embodiments have been described with reference to thefigures, it will be understood by those of ordinary skill in the artthat various changes in form and details may be made therein withoutdeparting from the spirit and scope as defined by the following claims.

What is claimed is:
 1. A fingerprint image preprocessing methodcomprising: receiving an input fingerprint image; performing ashort-time Fourier transform (STFT) on the input fingerprint image toobtain a transformed fingerprint image; comparing the input fingerprintimage and the transformed fingerprint image; and generating a combinedimage by combining the input fingerprint image and the transformedfingerprint image based on a result of the comparing, wherein thecomparing comprises comparing a pixel value of a pixel in the inputfingerprint image and a pixel value of a corresponding pixel in thetransformed fingerprint image, and wherein the generating comprisesapplying, to a corresponding pixel in the combined image, a result valueof combining the input fingerprint image and the transformed fingerprintimage or a pixel value of the transformed fingerprint image based on theresult of the comparing.
 2. The fingerprint image preprocessing methodof claim 1, wherein the generating further comprises: in response to apixel value of a pixel in an area in the input fingerprint image beinggreater than a pixel value of a corresponding pixel in an area in thetransformed fingerprint image, applying, to the corresponding pixel inthe combined image, the result value of combining the input fingerprintimage and the transformed fingerprint image.
 3. The fingerprint imagepreprocessing method of claim 1, wherein the generating furthercomprises: in response to a pixel value of a pixel in an area in theinput fingerprint image being less than a pixel value of a correspondingpixel in an area in the transformed fingerprint image, applying, to thecorresponding pixel in the combined image, the pixel value of thetransformed fingerprint image.
 4. The fingerprint image preprocessingmethod of claim 1, wherein the comparing further comprises: setting amask in each of the input fingerprint image and the transformedfingerprint image; and comparing each of a plurality of pixels in anarea in the input fingerprint image corresponding to the mask and eachof a plurality of pixels in an area in the transformed fingerprint imagecorresponding to the mask.
 5. The fingerprint image preprocessing methodof claim 4, wherein the comparing of each of the plurality of pixelscomprises: comparing a pixel value of a pixel in the area in the inputfingerprint image and a pixel value of a pixel in the area in thetransformed fingerprint image.
 6. The fingerprint image preprocessingmethod of claim 5, wherein the comparing of each of the plurality ofpixels further comprises: in response to a pixel value of a first pixelin the area in the input fingerprint image being greater than a pixelvalue of a second pixel corresponding to a position of the first pixelin the area in the transformed fingerprint image, allocating a firstidentifier to a corresponding position of the corresponding pixel in thecombined image; and in response to the pixel value of the first pixel inthe area in the input fingerprint image being less than the pixel valueof the second pixel corresponding to the position of the first pixel inthe area in the transformed fingerprint image, allocating a secondidentifier to the corresponding position of the corresponding pixel inthe combined image.
 7. The fingerprint image preprocessing method ofclaim 6, wherein the generating further comprises: applying, to thecorresponding pixel in the combined image, the result value of combiningthe input fingerprint image and the transformed fingerprint image basedon the first identifier; and applying, to the corresponding pixel in thecombined image, the pixel value of the transformed fingerprint imagebased on the second identifier.
 8. The fingerprint image preprocessingmethod of claim 1, wherein the comparing further comprises: comparing asensitivity of the input fingerprint image and a sensitivity of thetransformed fingerprint image for each corresponding area.
 9. Thefingerprint image preprocessing method of claim 8, wherein thegenerating further comprises: in response to the sensitivity of theinput fingerprint image being greater than the sensitivity of thetransformed fingerprint image, combining the input fingerprint image andthe transformed fingerprint image, and generating the combined imagebased on a result of the combining.
 10. A non-transitorycomputer-readable storage medium storing instructions that, whenexecuted by a processor, cause the processor to perform the fingerprintimage preprocessing method of claim
 1. 11. A fingerprint imagepreprocessing apparatus comprising: a processor; and a memory configuredto store an instruction to operate the processor, wherein the processoris configured to: receive an input fingerprint image; perform ashort-time Fourier transform (STFT) on the input fingerprint image toobtain a transformed fingerprint image; compare the input fingerprintimage and the transformed fingerprint image; generate a combined imageby combining the input fingerprint image and the transformed fingerprintimage based on a result of the comparing; wherein the processor isfurther configured to: compare a pixel value of a pixel in the inputfingerprint image and a pixel value of a corresponding pixel in thetransformed fingerprint image, and apply, to a corresponding pixel inthe combined image, a result value of combining the input fingerprintimage and the transformed fingerprint image or a pixel value of thetransformed fingerprint image based on the result of the comparing. 12.The fingerprint image preprocessing apparatus of claim 11, wherein theprocessor is further configured to: in response to a pixel value of apixel in an area in the input fingerprint image being greater than apixel value of a corresponding pixel in an area in the transformedfingerprint image, apply, to the corresponding pixel in the combinedimage, the result value of combining the input fingerprint image and thetransformed fingerprint image.
 13. The fingerprint image preprocessingapparatus of claim 11, wherein the processor is further configured to:in response to a pixel value of a pixel in an area in the inputfingerprint image being less than a pixel value of a corresponding pixelin an area in the transformed fingerprint image, apply, to thecorresponding pixel in the combined image, the pixel value of thetransformed fingerprint image.
 14. The fingerprint image preprocessingapparatus of claim 11, wherein the processor is further configured to:set a mask in each of the input fingerprint image and the transformedfingerprint image; and compare each of a plurality of pixels in an areain the input fingerprint image corresponding to the mask and each of aplurality of pixels in an area in the transformed fingerprint imagecorresponding to the mask.
 15. The fingerprint image preprocessingapparatus of claim 14, wherein the processor is further configured to:in response to a pixel value of a first pixel in an area in the inputfingerprint image being greater than a pixel value of a second pixelcorresponding to a position of the first pixel in an area in thetransformed fingerprint image, allocate a first identifier to acorresponding position of the corresponding pixel in the combined image;and in response to the pixel value of the first pixel in the area in theinput fingerprint image being less than the pixel value of the secondpixel corresponding to the position of the first pixel in the area inthe transformed fingerprint image, allocate a second identifier to thecorresponding position of the corresponding pixel in the combined image.16. The fingerprint image preprocessing apparatus of claim 13, whereinthe processor is further configured to: apply, to the correspondingpixel in the combined image, the result value of combining the inputfingerprint image and the transformed fingerprint image based on a firstidentifier; and apply, to the corresponding pixel in the combined image,the pixel value of the transformed fingerprint image based on a secondidentifier.
 17. A computing apparatus comprising: a fingerprint sensorconfigure to obtain an input fingerprint image; and a processorconfigured to perform a fingerprint verification based on the inputfingerprint image, wherein the processor is configured to: perform ashort-time Fourier transform (STFT) on the input fingerprint image toobtain a transformed fingerprint image; compare the input fingerprintimage and the transformed fingerprint image; generate a combined imageby combining the input fingerprint image and the transformed fingerprintimage based on a result of the comparing; wherein the processor isfurther configured to: compare a pixel value of a pixel in the inputfingerprint image and a pixel value of a corresponding pixel in thetransformed fingerprint image, and apply, to a corresponding pixel inthe combined image, a result value of combining the input fingerprintimage and the transformed fingerprint image or a pixel value of thetransformed fingerprint image based on the result of the comparing. 18.The computing apparatus of claim 17, wherein the processor is furtherconfigured to: in response to a pixel value of a pixel in an area in theinput fingerprint image being greater than a pixel value of acorresponding pixel in an area in the transformed fingerprint image,apply, to the corresponding pixel in the combined image, the resultvalue of combining the input fingerprint image and the transformedfingerprint image.
 19. The computing apparatus of claim 17, wherein theprocessor is further configured to: in response to a pixel value of apixel in an area in the input fingerprint image being less than a pixelvalue of a corresponding pixel in an area in the transformed fingerprintimage, apply, to the corresponding pixel in the combined image, thepixel value of the transformed fingerprint image.